Medical device with active brake release control

ABSTRACT

A computer-assisted medical device includes an articulated arm with a plurality of joints and a control unit coupled to the articulated arm. The control unit is configured to send a command to a plurality of brakes in the articulated arm to begin a release of the plurality of brakes in a predetermined staggered manner. In some embodiments the predetermined staggered manner prevents the simultaneous release of the plurality of breaks. In some examples, the predetermined staggered manner causes each brake in the plurality of brakes to release within a predetermined time of each other. In some embodiments, the first predetermined staggered manner causes each brake in the first plurality of brakes to release within a predetermined time of each other. In some embodiments, the first predetermined staggered manner causes each brake in the first plurality of brakes to begin a gradual release within a predetermined time of each other.

RELATED APPLICATIONS

This application is a U.S. National Stage patent application ofInternational Patent Application No. PCT/US2015/057671 (filed on Oct.27, 2015), the benefit of which is claimed, and claims priority to U.S.Provisional Patent Application No. 62/069,245 entitled “System andMethod for Integrated Operating Table,” which was filed Oct. 27, 2014,and U.S. Provisional Patent Application No. 62/134,225 entitled “Systemand Method for Reducing Tool Disturbances,” which was filed Mar. 17,2015, each of which are hereby incorporated by reference in theirentirety.

TECHNICAL FIELD

The present disclosure relates generally to operation of devices witharticulated arms and more particularly to reducing external disturbancesto an instrument pose.

BACKGROUND

More and more devices are being replaced with autonomous andsemiautonomous electronic devices. This is especially true in thehospitals of today with large arrays of autonomous and semiautonomouselectronic devices being found in operating rooms, interventionalsuites, intensive care wards, emergency rooms, and the like. Forexample, glass and mercury thermometers are being replaced withelectronic thermometers, intravenous drip lines now include electronicmonitors and flow regulators, and traditional hand-held surgicalinstruments are being replaced by computer-assisted medical devices.

These electronic devices provide both advantages and challenges to thepersonnel operating them. Many of these electronic devices may becapable of autonomous or semi-autonomous motion of one or morearticulated arms and/or end effectors. These one or more articulatedarms and/or end effectors each include a combination of links andarticulated joints that support motion of the articulated arms and/orend effectors. In many cases, the articulated joints are manipulated toobtain a desired position and/or orientation (collectively, a desiredpose) of a corresponding instrument located at a distal end of the linksand articulated joints of a corresponding articulated arm and/or endeffector. Each of the articulated joints proximal to the instrumentprovides the corresponding articulated arm and/or end effector with atleast one degree of freedom that may be used to manipulate the positionand/or orientation of the corresponding instrument. In many cases, thecorresponding articulated arms and/or end effectors may include at leastsix degrees of freedom that allow for controlling a x, y, and z position(collectively referred to as translational movement) of thecorresponding instrument as well as a roll, pitch, and yaw orientation(collectively referred to as rotational movement) of the correspondinginstrument. To provide for greater flexibility in control of the pose ofthe corresponding instrument, the corresponding articulated arms and/orend effectors are often designed to include redundant degrees offreedom. When redundant degrees of freedom are present it is possiblethat multiple different combinations of positions and/or orientations ofthe articulated joints may be used to obtain the same pose of thecorresponding instrument.

When devices with articulated arms are used for a medical procedure, itis not uncommon that one or more of the articulated arms may insert aninstrument and/or end effector into an opening of a patient. Dependingupon the procedure it may be desirable to release a lock and/or brake onone or more of the joints of the articulated arm in order to repositionat least a portion of the articulated arm. When the lock and/or brake isreleased, this may result in an undesirable movement in the positionand/or orientation of the articulated arm and more importantly a tip ofthe instrument and/or end effector that is positioned within thepatient. This undesirable movement may result in injury to a patient,injury to personnel in proximity to the articulated arms and/or endeffectors, damage to the articulated arms and/or end effectors, damageto other devices in proximity to the articulated arms and/or endeffectors, breach of a sterile field, and/or other undesirable outcomes.

Accordingly, it would be desirable to have the one or more joints in anarticulated arm correct for undesirable movement in an instrument, anarticulated arm, and/or an end effector when a brake and/or a lock isreleased in one or more joints of the articulated arm.

SUMMARY

Consistent with some embodiments, a computer-assisted medical deviceincludes a first articulated arm with a first plurality of joints and acontrol unit coupled to the first articulated arm. In some examples, thecontrol unit is configured to send a first command to a first pluralityof brakes in the first articulated arm to begin a release of the firstplurality of brakes in a first predetermined staggered manner.

Consistent with some embodiments, a method of controlling motion in amedical device includes sending a first command to a first plurality ofbrakes in a first articulated arm to begin a release of the firstplurality of brakes in a first predetermined staggered manner.

Consistent with some embodiments, a non-transitory machine-readablemedium comprising a plurality of machine-readable instructions whichwhen executed by one or more processors associated with a medical deviceare adapted to cause the one or more processors to perform a methodincluding sending a first command to a first plurality of brakes in afirst articulated arm to begin a release of the first plurality ofbrakes in a first predetermined staggered manner.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a simplified diagram of a computer-assisted system accordingto some embodiments.

FIG. 2 is a simplified diagram showing a computer-assisted systemaccording to some embodiments.

FIG. 3 is a simplified diagram of a kinematic model of acomputer-assisted medical system according to some embodiments.

FIG. 4 is a simplified diagram of a method of staggering the release ofbrakes on an articulated arm.

FIG. 5 is a simplified diagram of a method of compensating disturbancesfrom one joint set to a point of interest with a second joint set.

FIG. 6A is a simplified diagram illustrating a perspective view of anexemplary camera view and coordinate systems.

FIG. 6B is a simplified diagram illustrating a camera view from theperspective of a sensor or a display and the related coordinate systems.

FIG. 7 is a simplified diagram of a method of moving an articulated armbased on user commands while compensating for disturbances to one ormore joints in the articulated arm.

FIGS. 8A-8G are simplified schematic views that illustrate variouscomputer-assisted device system architectures that incorporate theintegrated computer-assisted device and movable surgical table featuresdescribed herein.

In the figures, elements having the same designations have the same orsimilar functions.

DETAILED DESCRIPTION

In the following description, specific details are set forth describingsome embodiments consistent with the present disclosure. It will beapparent to one skilled in the art, however, that some embodiments maybe practiced without some or all of these specific details. The specificembodiments disclosed herein are meant to be illustrative but notlimiting. One skilled in the art may realize other elements that,although not specifically described here, are within the scope and thespirit of this disclosure. In addition, to avoid unnecessary repetition,one or more features shown and described in association with oneembodiment may be incorporated into other embodiments unlessspecifically described otherwise or if the one or more features wouldmake an embodiment non-functional. The term “including” means includingbut not limited to, and each of the one or more individual itemsincluded should be considered optional unless otherwise stated.Similarly, the term “may” indicates that an item is optional.

FIG. 1 is a simplified diagram of a computer-assisted system 100according to some embodiments. As shown in FIG. 1, computer-assistedsystem 100 includes a device 110 with one or more movable or articulatedarms 120. Each of the one or more articulated arms 120 supports one ormore end effectors. In some examples, device 110 may be consistent witha computer-assisted surgical device. The one or more articulated arms120 each provides support for one or more instruments, surgicalinstruments, imaging devices, and/or the like mounted to a distal end ofat least one of the articulated arms 120. In some embodiments, device110 and the operator workstation may correspond to a da Vinci® SurgicalSystem commercialized by Intuitive Surgical, Inc. of Sunnyvale, Calif.In some embodiments, computer-assisted surgical devices with otherconfigurations, fewer or more articulated arms, and/or the like mayoptionally be used with computer-assisted system 100.

Device 110 is coupled to a control unit 130 via an interface. Theinterface may include one or more wireless links, cables, connectors,and/or buses and may further include one or more networks with one ormore network switching and/or routing devices. Control unit 130 includesa processor 140 coupled to memory 150. Operation of control unit 130 iscontrolled by processor 140. And although control unit 130 is shown withonly one processor 140, it is understood that processor 140 may berepresentative of one or more central processing units, multi-coreprocessors, microprocessors, microcontrollers, digital signalprocessors, field programmable gate arrays (FPGAs), application specificintegrated circuits (ASICs), and/or the like in control unit 130.Control unit 130 may be implemented as a stand-alone subsystem and/orboard added to a computing device or as a virtual machine. In someembodiments, control unit may be included as part of the operatorworkstation and/or operated separately from, but in coordination withthe operator workstation. Some examples of control units, such ascontrol unit 130 may include non-transient, tangible, machine readablemedia that include executable code that when run by one or moreprocessors (e.g., processor 140) may cause the one or more processors toperform the processes of method 400.

Memory 150 is used to store software executed by control unit 130 and/orone or more data structures used during operation of control unit 130.Memory 150 may include one or more types of machine readable media. Somecommon forms of machine readable media may include floppy disk, flexibledisk, hard disk, magnetic tape, any other magnetic medium, CD-ROM, anyother optical medium, punch cards, paper tape, any other physical mediumwith patterns of holes, RAM, PROM, EPROM, FLASH-EPROM, any other memorychip or cartridge, and/or any other medium from which a processor orcomputer is adapted to read.

As shown, memory 150 includes a motion control application 160 thatsupports autonomous and/or semiautonomous control of device 110. Motioncontrol application 160 may include one or more application programminginterfaces (APIs) for receiving position, motion, and/or other sensorinformation from device 110, exchanging position, motion, and/orcollision avoidance information with other control units regarding otherdevices, such as a surgical table and/or imaging device, and/or planningand/or assisting in the planning of motion for device 110, articulatedarms 120, and/or the end effectors of device 110. And although motioncontrol application 160 is depicted as a software application, motioncontrol application 160 may be implemented using hardware, software,and/or a combination of hardware and software.

In some embodiments, computer-assisted system 100 may be found in anoperating room and/or an interventional suite. And althoughcomputer-assisted system 100 includes only one device 110 with twoarticulated arms 120, one of ordinary skill would understand thatcomputer-assisted system 100 may include any number of devices witharticulated arms and/or end effectors of similar and/or different designfrom device 110. In some examples, each of the devices may include feweror more articulated arms and/or end effectors.

Computer-assisted system 100 further includes a surgical table 170. Likethe one or more articulated arms 120, surgical table 170 supportsarticulated movement of a table top 180 relative to a base of surgicaltable 170. In some examples, the articulated movement of table top 180may include support for changing a height, a tilt, a slide, aTrendelenburg orientation, and/or the like of table top 180. Althoughnot shown, surgical table 170 may include one or more control inputs,such as a surgical table command unit for controlling the positionand/or orientation of table top 180. In some embodiments, surgical table170 may correspond to one or more of the surgical tables commercializedby Trumpf Medical Systems GmbH of Germany.

Surgical table 170 is also coupled to control unit 130 via acorresponding interface. The interface may include one or more wirelesslinks, cables, connectors, and/or buses and may further include one ormore networks with one or more network switching and/or routing devices.In some embodiments, surgical table 170 may be coupled to a differentcontrol unit than control unit 130. In some examples, motion controlapplication 160 may include one or more application programminginterfaces (APIs) for receiving position, motion, and/or other sensorinformation associated with surgical table 170 and/or table top 180. Insome examples, motion control application 160 may contribute to motionplans associated with collision avoidance, adapting to and/or avoidrange of motion limits in joints and links, movement of articulatedarms, instruments, end effectors, surgical table components, and/or thelike to compensate for other motion in the articulated arms,instruments, end effectors, surgical table components, and/or the like,adjust a viewing device such as an endoscope to maintain and/or place anarea of interest and/or one or more instruments or end effectors withina field of view of the viewing device. In some examples, motion controlapplication 160 may plan and/or assist in the planning of motion forsurgical table 170 and/or table top 180. In some examples, motioncontrol application 160 may prevent motion of surgical table 170 and/ortable top 180, such as by preventing movement of surgical table 170and/or table top 180 through use of the surgical table command unit. Insome examples, motion control application 160 may help register device110 with surgical table 170 so that a geometric relationship betweendevice 110 and surgical table 170 is known. In some examples, thegeometric relationship may include a translation and/or one or morerotations between coordinate frames maintained for device 110 andsurgical table 170.

Control unit 130 may further be coupled to an operator workstation 190via the interface. Operator workstation 190 may be used by an operator,such as a surgeon, to control the movement and/or operation of thearticulated arms 120 and the end effectors. To support operation of thearticulated arms 120 and the end effectors, operator workstation 190includes a display system 192 for displaying images of at least portionsof one or more of the articulated arms 120 and/or end effectors. Forexample, display system 192 may be used when it is impractical and/orimpossible for the operator to see the articulated arms 120 and/or theend effectors as they are being used. In some embodiments, displaysystem 192 displays a video image from a video capturing device, such asan endoscope, which is controlled by one of the articulated arms 120, ora third articulated arm (not shown).

Operator workstation 190 includes a console workspace with one or moreinput controls 195 (also called “master controls 195”) that may be usedfor operating the device 110, the articulated arms 120, and/or the endeffectors mounted on the articulated arms 120. Each of the inputcontrols 195 may be coupled to the distal end of their own articulatedarms so that movements of the input controls 195 are detected by theoperator workstation 190 and communicated to control unit 130. Toprovide improved ergonomics, the console workspace may also include oneor more rests, such as an arm rest 197 on which operators may rest theirarms while manipulating the input controls 195. In some examples, thedisplay system 192 and the input controls 195 may be used by theoperator to teleoperate the articulated arms 120 and/or the endeffectors mounted on the articulated arms 120. In some embodiments,device 110, operator workstation 190, and control unit 130 maycorrespond to a da Vinci® Surgical System commercialized by IntuitiveSurgical, Inc. of Sunnyvale, Calif.

In some embodiments, other configurations and/or architectures may beused with computer-assisted system 100. In some examples, control unit130 may be included as part of operator workstation 190 and/or device110. In some embodiments, computer-assisted system 100 may be found inan operating room and/or an interventional suite. And althoughcomputer-assisted system 100 includes only one device 110 with twoarticulated arms 120, one of ordinary skill would understand thatcomputer-assisted system 100 may include any number of devices witharticulated arms and/or end effectors of similar and/or different designfrom device 110. In some examples, each of the devices may include feweror more articulated arms 120 and/or end effectors. Additionally, theremay be additional workstations 190 to control additional arms that maybe attached to device 110. Additionally, in some embodiments,workstation 190 may have controls for controlling surgical table 170.

FIG. 2 is a simplified diagram showing a computer-assisted system 200according to some embodiments. For example, the computer-assisted system200 may be consistent with computer-assisted system 100. As shown inFIG. 2, the computer-assisted system 200 includes a computer-assisteddevice 210 with one or more articulated arms and a surgical table 280.Although not shown in FIG. 2, the computer-assisted device 210 and thesurgical table 280 are coupled together using one or more interfaces andone or more control units so that at least kinematic information aboutthe surgical table 280 is known to the motion control application beingused to perform motion of the articulated arms of the computer-assisteddevice 210.

The computer-assisted device 210 includes various links and joints. Inthe embodiments of FIG. 2, the computer-assisted device is generallydivided into three different sets of links and joints. Starting at theproximal end with a mobile cart 215 (also called “patient-side cart 215”when configured to be placed patient-side sometimes during use) is aset-up structure 220. Coupled to a distal end of the set-up structure isa series of links and set-up joints 240 forming an articulated arm. Andcoupled to a distal end of the set-up joints 240 is a multi jointedmanipulator 260. In some examples, the series of set-up joints 240 andmanipulator 260 may correspond to one of the articulated arms 120. Andalthough the computer-assisted device is shown with only one series ofset-up joints 240 and a corresponding manipulator 260, one of ordinaryskill would understand that the computer-assisted device may includemore than one series of set-up joints 240 and corresponding manipulators260 so that the computer-assisted device is equipped with multiplearticulated arms.

As shown, the computer-assisted device 210 is mounted on the mobile cart215. The mobile cart 215 enables the computer-assisted device 210 to betransported from location to location, such as between operating roomsor within an operating room to better position the computer-assisteddevice in proximity to the surgical table 280. The set-up structure 220is mounted on the mobile cart 215. As shown in FIG. 2, the set-upstructure 220 includes a two part column including column links 221 and222. Coupled to the upper or distal end of the column link 222 is ashoulder joint 223. Coupled to the shoulder joint 223 is a two-part boomincluding boom links 224 and 225. At the distal end of the boom link 225is a wrist joint 226, and coupled to the wrist joint 226 is an armmounting platform 227.

The links and joints of the set-up structure 220 include various degreesof freedom for changing the position and orientation (i.e., the pose) ofthe arm mounting platform 227. For example, the two-part column is usedto adjust a height of the arm mounting platform 227 by moving theshoulder joint 223 up and down along an axis 232. The arm mountingplatform 227 is additionally rotated about the mobile cart 215, thetwo-part column, and the axis 232 using the shoulder joint 223. Thehorizontal position of the arm mounting platform 227 is adjusted alongan axis 234 using the two-part boom. And the orientation of the armmounting platform 227 may also be adjusted by rotation about an armmounting platform orientation axis 236 using the wrist joint 226. Thus,subject to the motion limits of the links and joints in the set-upstructure 220, the position of the arm mounting platform 227 may beadjusted vertically above the mobile cart 215 using the two-part column.The positions of the arm mounting platform 227 may also be adjustedradially and angularly about the mobile cart 215 using the two-part boomand the shoulder joint 223, respectively. And the angular orientation ofthe arm mounting platform 227 may also be changed using the wrist joint226.

The arm mounting platform 227 is used as a mounting point for one ormore articulated arms. The ability to adjust the height, horizontalposition, and orientation of the arm mounting platform 227 about themobile cart 215 provides a flexible set-up structure for positioning andorienting the one or more articulated arms about a work space locatednear the mobile cart 215 where an operation or procedure is to takeplace. For example, arm mounting platform 227 may be positioned above apatient so that the various articulated arms and their correspondingmanipulators and instruments have sufficient range of motion to performa surgical procedure on the patient. FIG. 2 shows a single articulatedarm coupled to the arm mounting platform 227 using a first set-up joint242. And although only one articulated arm is shown, one of ordinaryskill would understand that multiple articulated arms may be coupled tothe arm mounting platform 227 using additional first set-up joints.

The first set-up joint 242 forms the set-up joints 240 section of thearticulated arm that is the most proximal to the patient-side cart 215.The set-up joints 240 may further include a series of joints and links.As shown in FIG. 2, the set-up joints 240 may include links 244 and 246coupled via one or more joints (not expressly shown). The joints andlinks of the set-up joints 240 include the ability to rotate the set-upjoints 240 relative to the arm mounting platform 227 about an axis 252using the first set-up joint 242, adjust a radial or horizontal distancebetween the first set-up joint 242 and the link 246, adjust a height ofa manipulator mount 262 at the distal end of link 246 relative to armmounting platform 227 along an axis 254, and rotate the manipulatormount 262 about axis 254. In some examples, the set-up joints 240 mayfurther include additional joints, links, and axes permitting additionaldegrees of freedom for altering a pose of the manipulator mount 262relative to the arm mounting platform 227.

The manipulator 260 is coupled to the distal end of the set-up joints240 via the manipulator mount 262. The manipulator 260 includesadditional joints 264 and links 266 with an instrument carriage 268mounted at the distal end of the manipulator 260. An instrument 270 ismounted to the instrument carriage 268. The instrument 270 includes ashaft 272, which is aligned along an insertion axis. The shaft 272 istypically aligned so that it passes through a remote center of motion.Location of the remote center of motion 274 is typically maintained in afixed translational relationship relative to the manipulator mount 262so that operation of the joints 264 in the manipulator 260 result inrotations of the shaft 272 about the remote center of motion 274.Depending upon the embodiment, the fixed translational relation of theremote center of motion 274 relative to the manipulator mount 262 ismaintained using physical constraints in the joints 264 and links 266 ofthe manipulator 260, using software constraints placed on the motionspermitted for the joints 264, and/or a combination of both.Representative embodiments of computer-assisted surgical devices usingremote centers of motion maintained using physical constraints in jointsand links are described in U.S. patent application Ser. No. 13/906,888entitled “Redundant Axis and Degree of Freedom for Hardware-ConstrainedRemote Center Robotic Manipulator,” which was filed May 13, 2013, andrepresentative embodiments of computer-assisted surgical devices usingremote centers of motion maintained by software constraints aredescribed in U.S. Pat. No. 8,004,229 entitled “Software Center andHighly Configurable Robotic Systems for Surgery and Other Uses,” whichwas filed May 19, 2005, the specifications of which are herebyincorporated by reference in their entirety. In some examples, theremote center of motion 274 may correspond to a location of a bodyopening, such as an incision site or an orifice, in a patient 278.Because the remote center of motion 274 corresponds to the body opening,as the instrument 270 is used, the remote center of motion 274 remainsstationary relative to the patient 278 to limit stresses on the anatomyof the patient 278 at the remote center of motion 274. In some examples,the shaft 272 may be passed through a cannula (not shown) located at thebody opening. In some examples, instruments having a relatively largershaft or guide tube outer diameter (e.g., 4-5 mm or more) may be passedthrough the body opening using a cannula and the cannula may optionallybe omitted for instruments having a relatively smaller shaft or guidetube outer diameter (e.g., 2-3 mm or less).

At the distal end of the shaft 272 is an end effector 276. The degreesof freedom in the manipulator 260 due to the joints 264 and the links266 may permit at least control of the roll, pitch, and yaw of the shaft272 and/or end effector 276 relative to the manipulator mount 262. Insome examples, the degrees of freedom in the manipulator 260 may furtherinclude the ability to advance and/or withdraw the shaft 272 using theinstrument carriage 268 so that the end effector 276 may be advancedand/or withdrawn along the insertion axis and relative to the remotecenter of motion 274. In some examples, the manipulator 260 may beconsistent with a manipulator for use with the da Vinci® Surgical Systemcommercialized by Intuitive Surgical, Inc. of Sunnyvale, Calif. In someexamples, the instrument 270 may be an imaging device such as anendoscope, a gripper, a surgical tool such as a cautery or a scalpel,and/or the like. In some examples, the end effector 276 may includeadditional degrees of freedom, such as roll, pitch, yaw, grip, and/orthe like that allow for additional localized manipulation of portions ofthe end effector 276 relative to the distal end of the shaft 272.

During a surgery or other medical procedure, the patient 278 istypically located on the surgical table 280. The surgical table 280includes a table base 282 and a table top 284 with the table base 282being located in proximity to mobile cart 215 so that the instrument 270and/or end effector 276 may be manipulated by the computer-assisteddevice 210 while the shaft 272 of instrument 270 is inserted into thepatient 278 at the body opening. The surgical table 280 further includesan articulated structure 290 that includes one or more joints or linksbetween the table base 282 and the table top 284 so that the relativelocation of the table top 284, and thus the patient 278, relative to thetable base 282 is controlled. In some examples, the articulatedstructure 290 may be configured so that the table top 284 is controlledrelative to a virtually-defined table motion isocenter 286 that may belocated at a point above the table top 284. In some examples, isocenter286 may be located within the interior of the patient 278. In someexamples, isocenter 286 may be collocated with the body wall of thepatient at or near one of the body openings, such as a body opening sitecorresponding to remote center of motion 274.

As shown in FIG. 2, the articulated structure 290 includes a heightadjustment joint 292 so that the table top 284 may be raised and/orlowered relative to the table base 282. The articulated structure 290further includes joints and links to change both the tilt 294 andTrendelenburg 296 orientation of the table top 284 relative to theisocenter 286. The tilt 294 allows the table top 284 to be tiltedside-to-side so that either the right or left side of the patient 278 isrotated upward relative to the other side of the patient 278 (i.e.,about a longitudinal or head-to-toe axis of the table top 284). TheTrendelenburg 296 allows the table top 284 to be rotated so that eitherthe feet of the patient 278 are raised (Trendelenburg) or the head ofthe patient 278 is raised (reverse Trendelenburg). In some examples,either the tilt 294 and/or the Trendelenburg 296 rotations may beadjusted to generate rotations about isocenter 286. The articulatedstructure 290 further includes additional links and joints 298 to slidethe table top 284 along the longitudinal (cranial-caudal) axis relativeto the table base 282 with generally a left and/or right motion asdepicted in FIG. 2.

FIGS. 8A-8G are simplified schematic views that illustrate variouscomputer-assisted device system architectures that incorporate theintegrated computer-assisted device and movable surgical table featuresdescribed herein. The various illustrated system components are inaccordance with the principles described herein. In these illustrations,the components are simplified for clarity, and various details such asindividual links, joints, manipulators, instruments, end effectors, etc.are not shown, but they should be understood to be incorporated in thevarious illustrated components.

In these architectures, cannulas associated with one or more surgicalinstruments or clusters of instruments are not shown, and it should beunderstood that cannulas and other instrument guide devices optionallymay be used for instruments or instrument clusters having a relativelylarger shaft or guide tube outer diameter (e.g., 4-5 mm or more) andoptionally may be omitted for instruments having a relatively smallershaft or guide tube outer diameter (e.g., 2-3 mm or less).

Also in these architectures, teleoperated manipulators should beunderstood to include manipulators that during surgery define a remotecenter of motion by using hardware constraints (e.g., fixed intersectinginstrument pitch, yaw, and roll axes) or software constraints (e.g.,software-constrained intersecting instrument pitch, yaw, and roll axes).A hybrid of such instrument axes of rotation may be defined (e.g.,hardware-constrained roll axis and software-constrained pitch and yawaxes) are also possible. Further, some manipulators may not define andconstrain any surgical instrument axes of rotation during a procedure,and some manipulators may define and constrain only one or twoinstrument axes of rotation during a procedure.

FIG. 8A illustrates a movable surgical table 1100 and asingle-instrument computer-assisted device 1101 a are shown. Surgicaltable 1100 includes a movable table top 1102 and a table supportstructure 1103 that extends from a mechanically grounded table base 1104to support the table top 1102 at a distal end. In some examples,surgical table 1100 may be consistent with surgical table 170 and/or280. Computer-assisted device 1101 a includes a teleoperated manipulatorand a single instrument assembly 1105 a. Computer-assisted device 1101 aalso includes a support structure 1106 a that is mechanically groundedat a proximal base 1107 a and that extends to support manipulator andinstrument assembly 1105 a at a distal end. Support structure 1106 a isconfigured to allow assembly 1105 a to be moved and held in variousfixed poses with reference to surgical table 1100. Base 1107 a isoptionally permanently fixed or movable with reference to surgical table1100. Surgical table 1100 and computer-assisted device 1101 a operatetogether as described herein.

FIG. 8A further shows an optional second computer-assisted device 1101b, which illustrates that two, three, four, five, or more individualcomputer-assisted devices may be included, each having a correspondingindividual teleoperated manipulator and single-instrument assembly(ies)1105 b supported by a corresponding support structure 1106 b.Computer-assisted device 1101 b is mechanically grounded, and assemblies1105 b are posed, similarly to computer-assisted device 1101 a. Surgicaltable 1100 and computer-assisted devices 1101 a and 1101 b together makea multi-instrument surgical system, and they operate together asdescribed herein. In some examples, computer-assisted devices 1101 aand/or 1101 b may be consistent with computer-assisted devices 110and/or 210.

As shown in FIG. 8B, another movable surgical table 1100 and acomputer-assisted device 1111 are shown. Computer-assisted device 1111is a multi-instrument device that includes two, three, four, five, ormore individual teleoperated manipulator and single-instrumentassemblies as shown by representative manipulator and instrumentassemblies 1105 a and 1105 b. The assemblies 1105 a and 1105 b ofcomputer-assisted device 1111 are supported by a combined supportstructure 1112, which allows assemblies 1105 a and 1105 b to be movedand posed together as a group with reference to surgical table 1100. Theassemblies 1105 a and 1105 b of computer-assisted device 1111 are alsoeach supported by a corresponding individual support structure 1113 aand 1113 b, respectively, which allows each assembly 1105 a and 1105 bto be individually moved and posed with reference to surgical table 1100and to the one or more other assemblies 1105 a and 1105 b. Examples ofsuch a multi-instrument surgical system architecture are the da VinciSi® Surgical System and the da Vinci® Xi™ Surgical System,commercialized by Intuitive Surgical, Inc. Surgical table 1100 and asurgical manipulator system including an example computer-assisteddevice 1111 operate together as described herein. In some examples,computer-assisted device 1111 is consistent with computer-assisteddevices 110 and/or 210.

The computer-assisted devices of FIGS. 8A and 8B are each shownmechanically grounded at the floor. But, one or more suchcomputer-assisted devices may optionally be mechanically grounded at awall or ceiling and be permanently fixed or movable with reference tosuch a wall or ceiling ground. In some examples, computer-assisteddevices may be mounted to the wall or ceiling using a track or gridsystem that allows the support base of the computer-assisted systems tobe moved relative to the surgical table. In some examples, one or morefixed or releasable mounting clamps may be used to mount the respectivesupport bases to the track or grid system. As shown in FIG. 8C, acomputer-assisted device 1121 a is mechanically grounded at a wall, anda computer-assisted device 1121 b is mechanically grounded at a ceiling.

In addition, computer-assisted devices may be indirectly mechanicallygrounded via the movable surgical table 1100. As shown in FIG. 8D, acomputer-assisted device 1131 a is coupled to the table top 1102 ofsurgical table 1100. Computer-assisted device 1131 a may optionally becoupled to other portions of surgical table 1100, such as table supportstructure 1103 or table base 1104, as indicated by the dashed structuresshown in FIG. 8D. When table top 1102 moves with reference to tablesupport structure 1103 or table base 1104, the computer-assisted device1131 a likewise moves with reference to table support structure 1103 ortable base 1104. When computer-assisted device 1131 a is coupled totable support structure 1103 or table base 1104, however, the base ofcomputer-assisted device 1131 a remains fixed with reference to groundas table top 1102 moves. As table motion occurs, the body opening whereinstruments are inserted into the patient may move as well because thepatient's body may move and change the body opening locations relativeto the table top 1102. Therefore, for embodiments in whichcomputer-assisted device 1131 a is coupled to the table top 1102, thetable top 1102 functions as a local mechanical ground, and the bodyopenings move with reference to the table top 1102, and so withreference to the computer-assisted device 1131 a as well. FIG. 8D alsoshows that a second computer-assisted device 1131 b optionally may beadded, configured similarly to computer-assisted device 1131 a to createa multi-instrument system. Systems that include one or morecomputer-assisted device coupled to the surgical table operate asdisclosed herein.

In some embodiments, other combinations of computer-assisted deviceswith the same or hybrid mechanical groundings are possible. For example,a system may include one computer-assisted device mechanically groundedat the floor, and a second computer-assisted device mechanicallygrounded to the floor via the surgical table. Such hybrid mechanicalground systems operate as disclosed herein.

Inventive aspects also include single-body opening systems in which twoor more surgical instruments enter the body via a single body opening.Examples of such systems are shown in U.S. Pat. No. 8,852,208 entitled“Surgical System Instrument Mounting,” which was filed Aug. 12, 2010,and U.S. Pat. No. 9,060,678 entitled “Minimally Invasive SurgicalSystem,” which was filed Jun. 13, 2007, both of which are incorporatedby reference. FIG. 8E illustrates a teleoperated multi-instrumentcomputer-assisted device 1141 together with surgical table 1100 asdescribed above. Two or more instruments 1142 are each coupled to acorresponding manipulator 1143, and the cluster of instruments 1142 andinstrument manipulators 1143 are moved together by a system manipulator1144. The system manipulator 1144 is supported by a support assembly1145 that allows system manipulator 1144 to be moved to and fixed atvarious poses. Support assembly 1145 is mechanically grounded at a base1146 consistent with the descriptions above. The two or more instruments1142 are inserted into the patient at the single body opening.Optionally, the instruments 1142 extend together through a single guidetube, and the guide tube optionally extends through a cannula, asdescribed in the references cited above. Computer-assisted device 1141and surgical table 1100 operate together as described herein.

FIG. 8F illustrates another multi-instrument, single-body openingcomputer-assisted device 1151 mechanically grounded via the surgicaltable 1100, optionally by being coupled to table top 1102, table supportstructure 1103, or table base 1104. The descriptions above withreference to FIG. 8D also applies to the mechanical grounding optionsillustrated in FIG. 8F. Computer-assisted device 1151 and surgical table1100 work together as described herein.

FIG. 8G illustrates that one or more teleoperated multi-instrument,single-body opening computer-assisted devices 1161 and one or moreteleoperated single-instrument computer-assisted devices 1162 may becombined to operate with surgical table 1100 as described herein. Eachof the computer-assisted devices 1161 and 1162 may be mechanicallygrounded, directly or via another structure, in various ways asdescribed above.

FIG. 3 is a simplified diagram of a kinematic model 300 of acomputer-assisted medical system according to some embodiments. As shownin FIG. 3, kinematic model 300 may include kinematic informationassociated with many sources and/or devices. The kinematic informationis based on known kinematic models for the links and joints of acomputer-assisted medical device and a surgical table. The kinematicinformation is further based on information associated with the positionand/or orientation of the joints of the computer-assisted medical deviceand the surgical table. In some examples, the information associatedwith the position and/or orientation of the joints may be derived fromone or more sensors, such as encoders, measuring the linear positions ofprismatic joints and the rotational positions of revolute joints.

The kinematic model 300 includes several coordinate frames or coordinatesystems and transformations, such as homogeneous transforms, fortransforming positions and/or orientation from one of the coordinateframes to another of the coordinate frames. In some examples, thekinematic model 300 may be used to permit the forward and/or reversemapping of positions and/or orientations in one of the coordinate framesin any other of the coordinate frames by composing the forward and/orreverse/inverse transforms noted by the transform linkages included inFIG. 3. In some examples, when the transforms are modeled as homogenoustransforms in matrix form, the composing may be accomplished usingmatrix multiplication. In some embodiments, a system may use theDenavit-Hartenberg parameters and conventions for attaching coordinatereference frames to one or more points in the kinematic chain andtransforming from one reference frame to the other in the kinematicmodel 300. In some embodiments, the kinematic model 300 may be used tomodel the kinematic relationships of the computer-assisted device 210and the surgical table 280 of FIG. 2.

The kinematic model 300 includes a table base coordinate frame 305 thatis used to model a position and/or orientation of a surgical table, suchas surgical table 170 and/or surgical table 280. In some examples, thetable base coordinate frame 305 may be used to model other points on thesurgical table relative to a reference point and/or orientationassociated with the surgical table. In some examples, the referencepoint and/or orientation may be associated with a table base of thesurgical table, such as the table base 282. In some examples, the tablebase coordinate frame 305 may be suitable for use as a world coordinateframe for the computer-assisted system.

The kinematic model 300 further includes a table top coordinate frame310 that may be used to model positions and/or orientations in acoordinate frame representative of a table top of the surgical table,such as the table top 284. In some examples, the table top coordinateframe 310 may be centered about a rotational center or isocenter of thetable top, such as isocenter 286. In some examples, the z-axis of thetable top coordinate frame 310 may be oriented vertically with respectto a floor or surface on which the surgical table is placed and/ororthogonal to the surface of the table top. In some examples, the x- andy-axes of the table top coordinate frame 310 may be oriented to capturethe longitudinal (head to toe) and lateral (side-to-side) major axes ofthe table top. In some examples, a table base to table top coordinatetransform 315 is used to map positions and/or orientations between thetable top coordinate frame 310 and the table base coordinate frame 305.In some examples, one or more kinematic models of an articulatedstructure of the surgical table, such as articulated structure 290,along with past and/or current joint sensor readings is used todetermine the table base to table top coordinate transform 315. In someexamples consistent with the embodiments of FIG. 2, the table base totable top coordinate transform 315 models the composite effect of theheight, tilt, Trendelenburg, and/or slide settings associated with thesurgical table.

The kinematic model 300 further includes a device base coordinate framethat is used to model a position and/or orientation of acomputer-assisted device, such as computer-assisted device 110 and/orcomputer-assisted device 210. In some examples, the device basecoordinate frame 320 may be used to model other points on thecomputer-assisted device relative to a reference point and/ororientation associated with the computer-assisted device. In someexamples, the reference point and/or orientation may be associated witha device base of the computer-assisted device, such as the mobile cart215. In some examples, the device base coordinate frame 320 may besuitable for use as the world coordinate frame for the computer-assistedsystem.

In order to track positional (position related) and/or orientational(orientation related) relationships between the surgical table and thecomputer-assisted device, it is often desirable to perform aregistration between the surgical table and the computer-assisteddevice. As shown in FIG. 3, the registration may be used to determine aregistration transform 325 between the table top coordinate frame 310and the device base coordinate frame 320. In some embodiments, theregistration transform 325 may be a partial or full transform betweenthe table top coordinate frame 310 and the device base coordinate frame.The registration transform 325 is determined based on the architecturalarrangements between the surgical table and the computer-assisteddevice.

In the examples of FIGS. 8D and 8F, where the computer-assisted deviceis mounted to the table top 1102, the registration transform 325 isdetermined from the table base to table top coordinate transform 315 andknowing where the computer-assisted device is mounted to the table top112.

In the examples of FIGS. 8A-8C, 8E, and 8F, where the computer-assisteddevice is placed on the floor or mounted to the wall or ceiling,determination of the registration transform 325 is simplified by placingsome restrictions on the device base coordinate frame 320 and the tablebase coordinate frame 305. In some examples, these restrictions includethat both the device base coordinate frame 320 and the table basecoordinate frame 305 agree on the same vertical up or z-axis. Under theassumption that the surgical table is located on a level floor, therelative orientations of the walls of the room (e.g., perpendicular tothe floor) and the ceiling (e.g., parallel to the floor) are known it ispossible for a common vertical up or z axis (or a suitable orientationtransform) to be maintained for both the device base coordinate frame320 and the table base coordinate frame 305 or a suitable orientationtransform. In some examples, because of the common z-axis, theregistration transform 325 may model just the rotational relationship ofthe device base to the table top about the z-axis of the table basecoordinate frame 305 (e.g., a θz registration). In some examples, theregistration transform 325 may optionally also model a horizontal offsetbetween the table base coordinate frame 305 and the device basecoordinate frame 320 (e.g., a XY registration). This is possible becausethe vertical (z) relationship between the computer-assisted device andthe surgical table are known. Thus, changes in a height of the table topin the table base to table top transform 315 are analogous to verticaladjustments in the device base coordinate frame 320 because the verticalaxes in the table base coordinate frame 305 and the device basecoordinate frame 320 are the same or nearly the same so that changes inheight between the table base coordinate frame 305 and the device basecoordinate frame 320 are within a reasonable tolerance of each other. Insome examples, the tilt and Trendelenburg adjustments in the table baseto table top transform 315 may be mapped to the device base coordinateframe 320 by knowing the height of the table top (or its isocenter) andthe θz and/or XY registration. In some examples, the registrationtransform 325 and the table base to table top transform 315 may be usedto model the computer-assisted surgical device as if it were attached tothe table top even when this is architecturally not the case.

The kinematic model 300 further includes an arm mounting platformcoordinate frame 330 that may be used as a suitable model for a sharedcoordinate frame associated with the most proximal points on thearticulated arms of the computer-assisted device. In some embodiments,the arm mounting platform coordinate frame 330 may be associated withand oriented relative to a convenient point on an arm mounting platform,such as the arm mounting platform 227. In some examples, the centerpoint of the arm mounting platform coordinate frame 330 may be locatedon the arm mounting platform orientation axis 236 with the z-axisaligned with arm mounting platform orientation axis 236. In someexamples, a device base to arm mounting platform coordinate transform335 is used to map positions and/or orientations between the device basecoordinate frame 320 and the arm mounting platform coordinate frame 330.In some examples, one or more kinematic models of the links and jointsof the computer-assisted device between the device base and the armmounting platform, such as the set-up structure 220, along with pastand/or current joint sensor readings are used to determine the devicebase to arm mounting platform coordinate transform 335. In some examplesconsistent with the embodiments of FIG. 2, the device base to armmounting platform coordinate transform 335 may model the compositeeffect of the two-part column, shoulder joint, two-part boom, and wristjoint of the set up structure of the computer-assisted device.

The kinematic model 300 further includes a series of coordinate framesand transforms associated with each of the articulated arms of thecomputer-assisted device. As shown in FIG. 3, the kinematic model 300includes coordinate frames and transforms for three articulated arms,although one of ordinary skill would understand that differentcomputer-assisted devices may include fewer and/or more articulated arms(e.g., one, two, four, five, or more). Consistent with the configurationof the links and joints of the computer-assisted device 210 of FIG. 2,each of the articulated arms are modeled using a manipulator mountcoordinate frame, a remote center of motion coordinate frame, and aninstrument, end effector, or camera coordinate frame, depending on atype of instrument mounted to the distal end of the articulated arm.

In the kinematic model 300, the kinematic relationships of a first oneof the articulated arms is captured using a manipulator mount coordinateframe 341, a remote center of motion coordinate frame 342, an instrumentcoordinate frame 343, an arm mounting platform to manipulator mounttransform 344, a manipulator mount to remote center of motion transform345, and a remote center of motion to instrument transform 346. Themanipulator mount coordinate frame 341 represents a suitable model forrepresenting positions and/or orientations associated with amanipulator, such as manipulator 260. The manipulator mount coordinateframe 341 is typically associated with a manipulator mount, such as themanipulator mount 262 of the corresponding articulated arm. The armmounting platform to manipulator mount transform 344 is then based onone or more kinematic models of the links and joints of thecomputer-assisted device between the arm mounting platform and thecorresponding manipulator mount, such as the corresponding set-up joints240, along with past and/or current joint sensor readings of thecorresponding set-up joints 240.

The remote center of motion coordinate frame 342 is associated with aremote center of motion of the instrument mounted on the manipulator,such as the corresponding remote center of motion 274 of thecorresponding manipulator 260. The manipulator mount to remote center ofmotion transform 345 is then based on one or more kinematic models ofthe links and joints of the computer-assisted device between thecorresponding manipulator mount and the corresponding remote center ofmotion, such as the corresponding joints 264, corresponding links 266,and corresponding carriage 268 of the corresponding manipulator 260,along with past and/or current joint sensor readings of thecorresponding joints 264. When the corresponding remote center of motionis being maintained in fixed positional relationship to thecorresponding manipulator mounts, such as in the embodiments of FIG. 2,the manipulator mount to remote center of motion transform 345 includesan essentially static translational component and a dynamic rotationalcomponent that changes as the manipulator and instrument are operated.

The instrument coordinate frame 343 is associated with an end effectorlocated at the distal end of the instrument, such as the correspondingend effector 276 on corresponding instrument 270. The remote center ofmotion to instrument transform 346 is then based on one or morekinematic models of the links and joints of the computer-assisted devicethat move and/or orient the corresponding instrument and thecorresponding remote center of motion, along with past and/or currentjoint sensor readings. In some examples, the remote center of motion toinstrument transform 346 accounts for the orientation at which theshaft, such as the corresponding shaft 272, passes through the remotecenter of motion and the distance to which the shaft is advanced and/orwithdrawn relative to the remote center of motion. In some examples, theremote center of motion to instrument transform 346 may further beconstrained to reflect that the insertion axis of the shaft of theinstrument passes through the remote center of motion and accounts forrotations of the shaft and the end effector about the axis defined bythe shaft.

In the kinematic model 300, the kinematic relationships of a second oneof the articulated arms is captured using a manipulator mount coordinateframe 351, a remote center of motion coordinate frame 352, an instrumentcoordinate frame 353 (also called “instrument reference frame 353”), anarm mounting platform to manipulator mount transform 354, a mount toremote center of motion transform 355, and a remote center of motion toinstrument transform 356. The manipulator mount coordinate frame 351represents a suitable model for representing positions and/ororientations associated with a manipulator, such as manipulator 260. Themanipulator mount coordinate frame 351 is associated with a manipulatormount, such as the manipulator mount 262 of the correspondingarticulated arm. The arm mounting platform to manipulator mounttransform 354 is then based on one or more kinematic models of the linksand joints of the computer-assisted device between the arm mountingplatform and the corresponding manipulator mount, such as thecorresponding set-up joints 240, along with past and/or current jointsensor readings of the corresponding set-up joints 240.

The remote center of motion coordinate frame 352 is associated with aremote center of motion of the manipulator of an articulated arm, suchas the corresponding remote center of motion 274 of the correspondingmanipulator 260. The mount to remote center of motion transform 355 isthen based on one or more kinematic models of the links and joints ofthe computer-assisted device between the corresponding manipulator mountand the corresponding remote center of motion, such as the correspondingjoints 264, corresponding links 266, and corresponding carriage 268 ofthe corresponding manipulator 260, along with past and/or current jointsensor readings of the corresponding joints 264. When the correspondingremote center of motion is being maintained in fixed positionalrelationship to the corresponding manipulator mounts, such as in theembodiments of FIG. 2, the mount to remote center of motion transform355 includes an essentially static translational component that does notchange as the manipulator and instrument are operated and a dynamicrotational component that changes as the manipulator and instrument areoperated.

The instrument coordinate frame 353 is associated with a point on an endeffector, instrument, tool, and/or tool tip on an instrument mounted onthe articulated arm, such as the corresponding end effector 276 oncorresponding instrument 270. The remote center of motion to instrumenttransform 356 is then based on one or more kinematic models of the linksand joints of the computer-assisted device that move and/or orient thecorresponding instrument and the corresponding remote center of motion,along with past and/or current joint sensor readings. In some examples,the remote center of motion to instrument transform 356 accounts for theorientation at which the shaft, such as the corresponding shaft 272,passes through the remote center of motion and the distance to which theshaft is advanced and/or withdrawn relative to the remote center ofmotion. In some examples, the remote center of motion to instrumenttransform 356 may be constrained to reflect that the insertion axis ofthe shaft of the instrument passes through the remote center of motionand may account for rotations of the shaft and the end effector aboutthe insertion axis defined by the shaft.

In the kinematic model 300, the kinematic relationships of a third oneof the articulated arms is captured using a manipulator mount coordinateframe 361, a remote center of motion coordinate frame 362, a cameracoordinate frame 363, an arm mounting platform to manipulator mounttransform 364, a mount to remote center of motion transform 365, and aremote center of motion to camera transform 366. The manipulator mountcoordinate frame 361 represents a suitable model for representingpositions and/or orientations associated with a manipulator, such asmanipulator 260. The manipulator mount coordinate frame 361 isassociated with a manipulator mount, such as the manipulator mount 262of the corresponding articulated arm. The arm mounting platform tomanipulator mount transform 364 is then based on one or more kinematicmodels of the links and joints of the computer-assisted device betweenthe arm mounting platform and the corresponding manipulator mount, suchas the corresponding set-up joints 240, along with past and/or currentjoint sensor readings of the corresponding set-up joints 240.

The remote center of motion coordinate frame 362 is associated with aremote center of motion of the manipulator of the articulated arm, suchas the corresponding remote center of motion 274 of the correspondingmanipulator 260. The mount to remote center of motion transform 365 isthen based on one or more kinematic models of the links and joints ofthe computer-assisted device between the corresponding manipulator mountand the corresponding remote center of motion, such as the correspondingjoints 264, corresponding links 266, and corresponding carriage 268 ofthe corresponding manipulator 260, along with past and/or current jointsensor readings of the corresponding joints 264. When the correspondingremote center of motion is being maintained in fixed positionalrelationship to the corresponding manipulator mounts, such as in theembodiments of FIG. 2, the mount to remote center of motion transform365 includes an essentially static translational component that does notchange as the manipulator and instrument are operated and a dynamicrotational component that changes as the manipulator and instrument areoperated.

The camera coordinate frame 363 is associated with an imaging device,such an endoscope, mounted on the articulated arm. The remote center ofmotion to camera transform 366 is then based on one or more kinematicmodels of the links and joints of the computer-assisted device that moveand/or orient the imaging device and the corresponding remote center ofmotion, along with past and/or current joint sensor readings. In someexamples, the remote center of motion to camera transform 366 accountsfor the orientation at which the shaft, such as the corresponding shaft272, passes through the remote center of motion and the distance towhich the shaft is advanced and/or withdrawn relative to the remotecenter of motion. In some examples, the remote center of motion tocamera transform 366 may be constrained to reflect that the insertionaxis of the shaft of the imaging device passes through the remote centerof motion and accounts for rotations of the imaging device about theaxis defined by the shaft.

In some embodiments, an imaging device associated with camera coordinateframe 363 may stream video to an operator workstation such that a usermay view the video stream from camera coordinate frame 363. For example,the video captured by the imaging device may be relayed and displayed ondisplay system 192 of operator workstation 190 of FIG. 1. In someembodiments, the imaging device may be oriented such that it capturesvideo and/or images of an instrument associated with instrumentcoordinate frame 343 and/or an instrument associated with instrumentcoordinate frame 353. The instrument associated with instrumentcoordinate frame 343 and/or the instrument associated with instrumentcoordinate frame 353 may be operated by the user through a controller,such as input or master controls 195 of FIG. 1. In some embodiments, toallow for intuitive manipulation of the instruments and/or endeffectors, user commands from the controls may correlate with thecoordinate system of the camera coordinate frame 363. For example,commands of up and down, left and right, and in and out using thecontrollers may translate to movements of the instrument up and down,left and right, and in and out in relation to camera coordinate frame363. Up and down, left and right, in and out, may be resented by the x,y, and z translational axis of coordinate frame 363. Similarly, roll,pitch, and yaw commands may cause the instrument to roll, pitch, and yawin relation to the camera coordinate frame. In some embodiments, one ormore processors, such as processor 140 of FIG. 1, may translate usercommands from the camera coordinate frame 363 to respective commands andmotion in the instrument coordinate frames 343 and 353. Thetranslational commands may be through the kinematic relationships. Forexample, commands to the instrument associated with instrumentcoordinate frame 343 may go from camera coordinate frame 363 to remotecenter of motion reference frame 362 using transform 366, then fromremote center of motion reference frame 362 to mount coordinate frame361 using transform 365, mount coordinate frame 361 to arm mountingplatform coordinate frame 330 using transform 364, arm mounting platformcoordinate frame 330 to manipulator mount coordinate frame 341 usingtransform 344, manipulator mount coordinate frame 341 to remote centerof motion coordinate frame 342 using transform 345, and remote center ofmotion coordinate frame 342 to instrument coordinate frame 343 usingtransform 346. In this manner, any motion commands known in onereference frame can be transformed to corresponding commands in one ormore other coordinate frames.

As discussed above and further emphasized here, FIG. 3 is merely anexample which should not unduly limit the scope of the claims. One ofordinary skill in the art would recognize many variations, alternatives,and modifications. According to some embodiments, the registrationbetween the surgical table and the computer-assisted device may bedetermined between the table top coordinate frame 310 and the devicebase coordinate frame 320 using an alternative registration transform.When the alternative registration transform is used, registrationtransform 325 is determined by composing the alternative registrationtransform with the inverse/reverse of the table base to table toptransform 315. According to some embodiments, the coordinate framesand/or transforms used to model the computer-assisted device may bearranged differently dependent on the particular configuration of thelinks and joints of the computer-assisted device, its articulated arms,its end effectors, its manipulators, and/or its instruments. Accordingto some embodiments, the coordinate frames and transforms of thekinematic model 300 may be used to model coordinate frames andtransforms associated with one or more virtual instruments and/orvirtual cameras. In some examples, the virtual instruments and/orcameras may be associated with previously stored and/or latchedinstrument positions, projections of instruments and/or cameras due to amotion, reference points defined by a surgeon and/or other personnel,and/or the like.

As a computer-assisted system, such as computer-assisted systems 100and/or 200 are being operated, one of the goals is to minimize and/oreliminate the propagation of disturbances and/or movements from one ormore joints and/or links to the position of one or more points of aninstrument(s), link(s), and or joint(s). For example, referring to FIG.2, a disturbance to one or more of joints 242 and/or links 246 may causean injury to patient 278 if the disturbance propagated to end effector276 (end effector 276 being an exemplary point of interest) while insideof patient 278.

In one mode of operation for the computer-assisted system, one or morejoints of the surgical table and joints of the articulated arms may belocked and/or held in place through the use of servo control and/orbrakes so that motion of the joints is limited and/or prohibitedentirely. In some examples, this may allow the joints of themanipulators to control an instrument undisturbed by motion from otherjoints when accomplishing a desired procedure. In some embodiments, themanipulators may be physically constrained to maintain a remote centerof motion and motion of one or more joints that do not make up themanipulator might undesirably cause the remote center of motion to move.In those examples, it may be beneficial to have the joints that do notmake up the manipulators be locked in place through physical and/orservo control braking systems. However, there may be instances whereallowing movement to the remote center of motion would be desirable, andthus allowing for release of the brakes locking one or more of thejoints that may affect the position of the remote center of motion.

In some examples, the instruments may be inserted through a body openingof a patient during a procedure. In some examples, the position of theinstruments may be controlled via teleoperation by a surgeon at anoperator console such as workstation 190 of FIG. 1. It may, however, bedesirable to support other modes of operation for the computer-assistedsystem that allow for movement in the articulated arms while theinstruments remain inserted through a body opening of the patient. Theseother modes of operation may introduce risks that are not present inmodes of operation when the instruments are not inserted into a bodyopening of the patient. In some examples, these risks may include but atnot limited to injury to the patient when the instruments are allowed tomove relative to the patient, the instruments breaching a sterile field,damage from collisions between the articulated arms, and/or the like.

In a general case, these other modes of operation may be characterizedby a goal of maintaining a point of an instrument inserted into anopening of a patient relative to a patient when one or more jointsproximal to the instrument are subject to a disturbance that results ina change to positions and/or orientations (i.e., movements) of the oneor more joints. Because disturbances in one or more first or disturbedjoints proximal to an instrument results in a change in the position ofthe instrument, it may be desirable to introduce movement in one or moresecond or compensating joints that compensate for the movement of theinstrument caused by the movement of the disturbed joints. Determiningthe extent of the disturbance and the amount of compensation depends onthe type and nature of the disturbance, such as whether the disturbanceis associated with movement of the surgical table or patient, or whetherthe disturbance is confined to the articulated arm used to control theinstrument.

In one category of these other modes of operation is when the patient isnot moving so that the position of the instrument and/or a point on theinstrument may be monitored and maintained in any suitable worldcoordinate frame. This may include disturbances associated withcontrolled motion of the articulated arm. In some examples, thecontrolled motion of the articulated arm may include movement in one ormore joints used to set-up the articulated arm and/or the manipulatorbefore performing a procedure. One example of this includes the movementof one or more of the set-up structures of a computer-assisted deviceconsistent with the embodiments of FIG. 2 where the arm mountingplatform 227 is translated and aligned to allow the set-up joints 240 bemoved to provide good range of motion in the manipulator 260 during aprocedure. Examples of this type of motion is described in greaterdetail in U.S. Provisional Patent Application No. 61/954,261 entitled“System and Method for Aligning with a Reference Target,” which wasfiled on Mar. 17, 2014 and which is hereby incorporated by reference.This category may further include disturbances associated with therelease of brakes and/or other joint locks prior to initiating othermotion. In some examples, external forces and/or torques on the shaft ofthe instrument, such as due to forces and torques applied to theinstrument by the body wall of the patient while inserted into thepatient, may result in undesirable motion of the instrument when thebrakes and/or locks are released and forces and/or torques are absorbedby the released joints. This category may further include disturbancescaused by operation of the articulated arm in a clutched or float statesuch as might occur during manual repositioning of the articulated armby an operator and/or due to a collision between the articulated arm andan obstacle. Examples of this type of motion are described in greaterdetail in U.S. Provisional Patent Application No. 91/954,120 entitled“System and Method for Breakaway Clutching in an Articulated Arm,” whichwas filed on Mar. 17, 2014 and which is hereby incorporated byreference. Additional examples of this type of motion may occur when thecomputer-assisted device prepares for integrated surgical table motionby releasing one or more brakes or locks and the forces and torquesapplied to the instruments inserted into a body opening by the body wallare released. These types of motions and disturbances are described ingreater detail in U.S. Provisional Patent Application No. 62/134,207entitled “System and Method for Integrated Surgical Table,” which wasfiled Mar. 17, 2015 concurrently filed PCT Patent Application No.PCT/US2015/057656 entitled “System and Method for Integrated SurgicalTable” and published as WO2016/069648 A1, both of which are herebyincorporated by reference in their entirety.

With respect to brake releases on one or more joints of an articulatedarm, any forces and/or torques being applied to the joint upon the brakerelease may cause motion to the joints and their respective links. Thesedisturbances may often cause quick and sometimes large jumps inmovements to the end effectors and/or instruments attached to thearticulated arms. Though any single disturbance from a single brakerelease may be small, the combined disturbances when brakes for multiplejoints are released simultaneously may be rather large. These largejumps in movement may cause the end effectors to injure the patient.Furthermore, these jumps in movement are often too quick for humanreactions, and therefore, difficult, if not impossible, to remedythrough manual control. One method to reduce the jumps and provide usersthe ability to react would be to slowly reduce the braking force foreach brake over time and/or release the brakes one at a time. However,during surgical operations, it is important to minimize any unnecessarytime consumption because mortality rates of a patient during surgery goup in relation to the length of time the patient is in surgery.Therefore, it is desirable to release brakes within a short period oftime (over a few seconds or less).

FIG. 4 is a simplified diagram of an exemplary method 400 for astaggered brake release according to some embodiments. In some examples,method 400 may be used to stagger the release of one or more brakes onjoints of one or more articulated arms, such as articulated arms 120 ofFIG. 1. According to some embodiments, method 400 may include one ormore of the processes 410-430 which may be implemented, at least inpart, in the form of executable code stored on a non-transitory,tangible, machine readable media that when run on one or more processors(e.g., the processor 140 in control unit 130 of FIG. 1) may cause theone or more processors to perform one or more of the processes 410-430.

At process 410, the number of articulated arms for brake release isdetermined. In some embodiments, the number of articulated arms may bepredetermined. For example, the device may be hard coded with a specificnumber of articulated arms. In some embodiments, a user may set thenumber of articulated arms set for brake releases. For example, usingbuttons and/or switches on a work station, such as workstation 190 ofFIG. 1, a user and/or operator may be able to select which arms and/orthe number of arms that will have brakes released. In some embodiments,the number of articulated arms may be detected by connections to one ormore ports and/or other communications interfaces. For example, acomputer aided system, such as computer aided system 100 of FIG. 1, mayhave a brake release mode for releasing one or more brakes in thearticulated arms and control unit 130 may determine the number ofarticulated arms controlled by the system through the communicationinterfaces with the articulated arms.

A process 420, the timing of brake releases for each articulated arm isdetermined. In some embodiments, the timing of each brake release may bestaggered to ensure no single brake is released at the same time asanother brake. In some embodiments, the joints on an articulated arm maybe set to automatically release brakes in quick succession and a centralcontroller, such as control unit 130 of FIG. 1, may determine how tostagger the start of each brake release for each arm. For example, anarticulated arm, such as articulated arm 120 of FIG. 2 may have brakesbeing released for a set of joints, such as setup joints 240 of FIG. 2.Assuming there are four setup joints, these joints may be released inquick succession, such as released every 0.25 seconds. The time betweenrelease of different brakes and the order in which brakes are releasedmay be preset and automatic once instructions for brake release arereceived. This allows for quick reliable brake releases without thedelays of communication and processing times.

To ensure that no brake release for any single articulated arm isreleased at the same time as a brake release for another articulatedarm, the command to begin brake release for each arm is sent at acalculated interval that prevents simultaneous brake release. Forexample, each arm may be ordered to begin brake release 0.25 secondsfrom each other when the interval between brake releases in eacharticulated arm is 0.2 seconds. In this example, the four jointsreleased in the first arm will be released at time 0 s, 0.2 s, 0.4 s,and 0.6 s. The release of the brakes for the next arm and the fourjoints of the next arm would be 0.25 s, 0.45 s, 0.65 s, and 0.85 s. Thethird arm would release brakes at 0.5 s, 0.7 s, 0.9 s, and 1.1 s.Finally, the fourth arm would release at 0.75 s, 0.95 s, 1.15 s, and1.35 s. One of ordinary skill in the art would recognize that there aremany possibilities for brake release intervals and intervals betweencommands such that no brake is released at the same time, all of whichare contemplated herein. One simple exemplary method of determining wheneach arm begins brake release would be to divide the time intervalbetween brake releases on an arm by the number of arms.

In some embodiments, the order of brake release is predetermined. Forexample, the brakes may be released in the order of brakes for jointsthat have the least motion to the brakes for joints that have the mostmotion. In some embodiments, determining which joints cause or move themost during a brake release may be determined through empirical tuning.Based on experimentation, releasing brakes for rotational joints and/ortranslational joints for motion parallel to the floor (sometimesreferred to as joints with horizontal motion) tend to cause the leastamount of motion and releasing brakes for joints that allow movementperpendicular to the floor (sometimes referred to as joints withvertical motion) tend to cause the most motion. In some embodiments, aforce sensor may indicate which brakes are bearing the most force and/ortorque and determine that those joints will move the most. In someembodiments, the order of brake release may be determined based on theconfiguration and position of each joint. For example, joints at the endof its range of motion may be less likely to move when a brake for thejoint is released

In some embodiments, an articulated arm may not be in communicationswith any other articulated arm. Thus, when there are brakes releasecommands sent to one or more arms is delayed, this may cause one or morejoints to be released simultaneously. In some examples, to ensuresimultaneous brakes releases don't occur, each robotic arm may share aglobal clock cycle count for the entire system and each arm may be givena window of time for which brakes can be released. For example, if asystem were to have the brake release of all arms to begin within a onesecond time frame, a 1 kHz processor would have 1000 clock cycles withinthat time frame. If there were four articulated arms, the clock cycleswithin one second can be divided into four windows of 250 cycles. Cycles0-249 can be designated for one arm, 250-499 for the second arm, 500-749for the third arm, and 750-999 for the fourth arm. The timing window canthen be determined by the modulo of the global clock cycle by 1000. Inthis manner, each window is repeated once every 1000 clock cycles. Whenan arm misses a window for brake release in a clock cycle, the brakerelease for that articulated arm will release when the 250 clock cyclewindow for that articulated arm repeats.

More generally, windows of time for brake release is determined bydividing the number of clock cycles for a time limit by the number ofarms. The brake release window for a given global clock time isdetermined by using the modulo of the global clock by the number ofclock cycles for the time limit. In some embodiments, buffers may beadded to each clock cycle window to prevent brake releases occurringwithin one clock cycle of each other. For example, based on the 250clock cycle window, the window for brake release may be a single clockcycle for each arm at 249, 499, 749 and 999. In this manner there is a249 clock cycle buffer between when brake release begins for eacharticulated arm, or approximately 0.25 seconds based on the 1 kHzprocessor.

In some embodiments, the central controller directly determines whichbrake is released, the order of the brake releases, and when. In thismanner, the central control can ensure that none of the brakes arereleased at the same time. In some embodiments the brakes may begradually released over time. However, method 400 is set up to also workwith binary brakes without a gradual reduction in braking force. Thisallows the method to be used with cheaper less complex brakes that arepresent in legacy systems. Furthermore, binary brakes are desirable forbeing cheaper, more reliable, and simpler.

At process 430, the brakes to joints of the articulated arms arereleased in accordance with the brake release timing set up at process420.

In some embodiments, the joints of the articulated arms may have brakesthat may be gradually released. In some examples each of the brakescould be concurrently released gradually over time. In some embodiments,brakes may be released in accordance with the timing determined atprocess 420 with the brakes gradually releasing within an allottedtiming window and/or with gradual brake release beginning at thebeginning of the allotted timing window. In some examples, the gradualrelease of the brakes may be accomplished by ramping a signal thatcontrols braking force over time. In some examples, the ramped signalmay be a voltage, a current, a pulse-width duty cycle, and/or the like.Depending of a transfer relationship between the ramped signal and thebraking force, the change in value of the ramped signal over time may belinear and/or non-linear.

FIG. 5 is a simplified diagram of a method 500 for compensating fordisturbances, such as disturbances caused by a brake release accordingto some embodiments. In some examples, the disturbances caused by therelease of brakes for one or more joints, collectively referred to as afirst set of joints or disturbed joints, may be compensated for by oneor more other joints, collectively referred to as a second set of jointsor compensating joints, such that a point associated with an articulatedarm (point of interest) and/or instrument is minimally or completelyunaffected by the disturbance. The point of interest may be the locationof a particular joint, remote center of motion, instrument, endeffector, tool, a tip of a tool, a point along a kinematic chain, anapproximation of any of the preceding points, and/or the like. In someembodiments, the compensating joints may provide one or more redundantdegrees of freedom that are used to compensate for movement caused bythe changes in the disturbed joints.

In some examples the disturbed joints may all be distal or proximal tothe point of interest in comparison to the compensating joints. Forexample, the series of set-up joints 240 of FIG. 2 may be the disturbedjoints, joints 264 may be the compensating joints, and end effector 276may be the point of interest. In this example, set-up joints 240 aremore distal from end effector 276 than joints 264 of manipulator 260,which may be the compensating joints.

In other examples some of the disturbed joints may be betweencompensating joints. In these examples, however, the kinematic chain maybe broken into subsets of joints and connectors such that each subsethas all the disturbed joints in the subset as proximal or distal to asubset point of interest (which may or may not be the same point ofinterest for the entire kinematic chain) in comparison to thecompensating joints within the subset. In this manner, the model forcompensating disturbed joints all being more distal or proximal to apoint of interest in relation to all compensating joints can be appliedto kinematic chains where some of the disturbed joints are more proximaland some of the disturbed joints are more distal to a point of interestin relation to one or more compensating joints by breaking the kinematicchain into sub-kinematic chains.

Method 500 is discussed in the context of embodiments where all of thedisturbed joints are more proximal to the point of interest than thecompensating joints in the kinematic chain. However, one of ordinaryskill in the art would understand that this method is applicable tosituations where all of the disturbed joints are more distal rather thanproximal to the compensating joints. Furthermore, as discussed above,this method can also be applied in situations where the disturbed jointsare interleaved among the compensating joints in a kinematic chain bytreating the entire kinematic chain as a collection of sub-kinematicchains wherein each sub-kinematic chain may be chosen such that all ofthe disturbed joints are proximal or distal to the compensating jointsin relation to one or more points of interest.

Method 500 may include one or more of the processes 510-560 which may beimplemented, at least in part, in the form of executable code stored ona non-transitory, tangible, machine readable media that when run on oneor more processors (e.g., the processor 140 in control unit 130 ofFIG. 1) may cause the one or more processors to perform one or more ofthe processes 510-560.

In some embodiments, method 500 may be used to compensate for changes inthe position of the instrument due to motion in one or more disturbedjoints by introducing compensating motion in one or more compensatingjoints. In some examples, method 500 may be used when the motion in thedisturbed joints is due to controlled motion, clutched motion, brake orlock release, and/or the like. In some examples, method 500 may be usedwhen motion in the disturbed joints is due to the release of a brake,such as during process 430. In some examples consistent with theembodiments of FIG. 2, the one or more disturbed joints and/or the oneor more compensating joints may include any of the joints in set-upstructure 220, the set-up joints 240, and/or any joints of manipulator260 proximal to the instrument. In some embodiments, use of method 500may be limited to operation when an instrument, cannula, and/or the likeis coupled to the distal end of a corresponding articulated arm, endeffector, and/or manipulator so that a remote center of motion for thearticulated arm, end effector, and/or manipulator may be defined. Insome embodiments, method 500 may include having the pose and/or positionof the instrument at least partially maintained using resistance fromthe wall of an orifice or incision of a patient and/or by an operator ofthe computer-assisted device. In some embodiments, method 500 may beapplied in conjunction with and/or on top of movement commands from anoperator of the computer-assisted device such that the operator stillhas control over the movement of one or more instruments.

At a process 510, a reference coordinate frame is established. For easeof calculations, process 510 may use a non-moving/fixed point in akinematic chain as a reference frame. For example, with respect to FIGS.2 and 3, if setup joints 240 are the disturbed joints, and the joints ofmanipulator 260 are the compensating joints, any point more proximal tocart 215 than the setup joints 240 could be used as a reference frame,including joint 226 which may be the arm mounting platform coordinateframe 330 of FIG. 3.

At a process 520, a reference transform from the reference coordinateframe to a point for steadying, such as end effector 276 of FIG. 2, isestablished. This reference transform may be established prior to anydisturbances to the disturbed joints. In some embodiments, thistransform may be a transform for a particular position of one or morejoints, instruments, links, and/or any other object in a kinematicchain. In some embodiments, the reference transform may be made up ofseveral sub-transforms between several coordinate frames. For example,the transform from arm mounting platform coordinate frame 300 toinstrument coordinate frame 353 is made up of transforms 354, 355, and356. In some embodiments, these transforms may be stored in memory, suchas memory 150 of FIG. 1.

At process 530, the disturbances in the joints are detected and aprediction of the movement to the point of interest caused by thedisturbance is determined using the new position of the disturbedjoints.

The following is an exemplary method of determining a disturbance to apoint of interest caused by the disturbance in the disturbed joints. Ata first step, a transform between two coordinate frames spanning thedisturbed joints before disruption may be stored in memory, such asmemory 150 of FIG. 1. In some examples, the two coordinate frames spanthe disturbed joints when a first one of the coordinate frames isproximal to the most proximal of the disturbed joints and the second oneof the coordinate frames is distal to the most distal of the disturbedjoints. In a second step, a transform between two coordinate framesspanning the compensating joints may be stored in memory. In a thirdstep, a location for an undisturbed point of interest is determinedusing the saved transforms in the first and second steps. In someexamples, this location of the point of interest may model the point ofinterest before any disturbed or compensating motion takes place. In afourth step, an estimated location for the point of interest isdetermined by using a live/current transform between the two coordinateframes spanning the disturbed joints and the saved transform in thesecond step. In some examples, an estimated location of the point ofinterest that accounts for the changes in the disturbed joints, but notthe compensating joints may be determined. In a fifth step, thedifference between the location for the undisturbed point of interestand the estimated point of interest can be used to determine how themotion of the disturbed joints has moved the point of interest. As wouldbe recognized by one of ordinary skill in the art, the stored transformbetween the two coordinate frames spanning the compensating joints maybe stored in any suitable data structure capable of representing ahomogenous transform, such as in the form of a matrix, and/or the like.In some examples, the transform may be stored as joint angles and/orpositions from which the transform may be recomputed using one or morekinematic models of the compensating joints. The stored transform may beconducted for a particular configuration and/or a configuration for aparticular point in time. An exemplary application of this method toFIG. 3 may include the following, joint motions caused by a disturbancemay change transform 354 but not transforms 355 and 356. In this manner,a prediction of the change in position caused by the disturbance can bedetermined by using the changed transform 354 and the saved transforms355 and 356. Furthermore, this allows isolation of the motion of theinstrument reference frame 353 caused by the disturbance from actuatedmotion caused by user commands. Therefore, even though saved transforms355 and 356 may not be used for determining the actual position of aninstrument, it can be used to predict a motion to a point of interestcaused by a disturbance.

At an optional process 540, the prediction of at 530 is adjusted forreal world errors. In a perfect world, where all links between jointsare perfectly ridged, the prediction at process 530 would be a perfectmatch to where the point of interest moved due to the disturbances inthe joints. However, the links between joints bend and give depending onthe amount of force applied to the links and the flexural strength ofthe link material. For example, during a surgical operation the skin ofthe operating subject will often tent or ride up the cannula enteringthe subject. This tenting will apply a force onto the cannula, which inturn applies a force onto the links and joints of the device. While thebrakes are engaged, the skin is held up by the device to maintain itsposition, but when the brakes are released for some of the joints, thosejoints will be allowed to move freely and be disturbed. In turn, thetented skin will move the released joints until the skin is no longerapplying a force in the direction of movement of the joint. Because theskin is no longer applying a force (or applying less force) on thecannula the links between joints that were bearing forces caused by thetenting will de-flex. The de-flexing will often counteract some of themotion by the joints, thus causing movement predictions based on jointmovements to be an over estimate. This may cause an error in theprediction at process 530. At process 540, adjustments are made toaccount for this source of error and other sources of errors to theprediction.

In some embodiments, error correction may be a scalar multipleestimating the amount of error in the translational motion predictedduring process 530. The estimates may depend on one or more factors suchas the patient, procedure, and/or orientation of the articulated device.For example, there may be settings for child, adult, veterinary (e.g.animal species), area of surgery (e.g. arm, leg, stomach, chest,cranial, etc.), and or the like. In some examples, a general errorestimate may be used for all cases. In some embodiments, a single errorestimate between 80-95% (e.g., 85%) of the translational motion and 100%of the rotational motion predicted during process 530 for thedisturbance at the point of interest may be used as the error correctedposition of the point of interest. For ease of calculations, the errorcorrection may be computed at the coordinate frame of the point ofinterest. In this manner, corrections to translational motion on thepoint of interest may be treated differently than rotational errors(e.g., using one fraction of the predicted translational disturbance andanother fraction for the predicted rotational disturbance). Though thecorrections may be applied at a different reference frames, thecomputations may become difficult because the rotation in anothercoordinate frame contributes to translational motion in the coordinateframe of the point of interest. In some embodiments, when process 540 isomitted, the prediction determined during process 530 may be used as theerror corrected prediction. Using the prediction determined duringprocess 530 may, however, introduce a slight over correction.

At a process 550, differences between the error corrected predictedtransform and the reference transform are determined. The differencesbetween the error corrected predicted transform determined duringprocess 540 and the reference transform determined during process 520represent errors that are being introduced into the point of interest bythe disturbance. Unless the errors are compensated for by movement usingone or more compensating joints of the articulated arm, the placement ofthe point of interest may undesirably change. In some examples, thedifferences may be determined by multiplying corresponding matricesand/or vector representations of the actual and reference transforms. Insome examples, the differences may be represented as an error transformdetermined by composing an inverse/reverse of the reference transformwith the error corrected prediction transform.

At a process 560, compensating joint changes are determined based on thedifferences. Using the differences between the actual transform and thereference transform determined during process 550, changes in the one ormore compensating joints are determined. The differences between theerror corrected predicted transform and the reference transform ismapped from the reference coordinate system of the reference transformsto one or more local coordinate systems associated with each of thecompensating joints. In effect, this transforms the errors in theplacements of the point of interest from the reference coordinate systemto relative errors of the point of interest relative to the compensatingjoints. In some examples, one or more kinematic models are used totransform the differences to the local coordinate systems. In someexamples, the compensating joints may include any of the joints of thearticulated arm and/or the manipulator that are not one of the disturbedjoints. Once the relative errors of the point of interest aredetermined, they may be used to determine the movements for each of thecompensating joints. In some examples, an inverse Jacobian may be usedto map the relative errors to the movements of the compensating joints.In some examples, the movements in the compensating joints may beapplied as joint velocities applied to the compensating joints.

At a process 570, the compensating joints are driven. One or morecommands are sent to the one or more actuators in the compensatingjoints based on the movements of the compensating joints determinedduring process 560. The commands sent to the compensating joints correctfor the disturbances to the point of interest introduced by themovements in the one or more disturbed joints so that placement of thepoint of interest in the reference coordinate system is maintained withlittle or no disturbance. As long as the one or more compensating jointscontinue to makes corrective changes to the placement of the point ofinterest, processes 530-570 may be repeated to compensate for any errorsintroduced into the position and placement of the point of interest.

According to some embodiments, correcting, driving, or moving the pointof interest may be conducted from a reference point different from thepoint of interest. This may allow for simpler computations and/or reuseof functions and/or algorithms for driving joint movements, such asjoint positioning and velocity. For example, referring to FIG. 2, it maybe computationally easier to have joints of the manipulator 260 ofcomputer-assisted system 200 adjust for errors at manipulator mount 262than at end effector 276. In some examples, a system implementing method500 may create a reference position for a different reference point thatencompasses the error adjusted prediction determined during process 540.This reference point may then be used to drive the compensating jointsto adjust for the disturbances. This works because the disturbance ofthe point of interest may be represented by a disturbance at otherpoints in a kinematic chain, such as the reference point. The referenceposition may be determined using one or more reference transforms, suchas the reference transforms established during process 520. In someinstances, the inverse of the reference transforms may be used. Inaccordance with FIG. 2, adjusting for movement of end effector 276 mayencompass creating a reference manipulator mount 262 position based onan the error corrected predicted position of end effector 276 that maybe caused by a disturbances to set up joints 240 during release of oneor more brakes on the set up joints 240.

According to some embodiments, process 570 may be subject to practicallimitations. In some examples, the ability of one or more of thecompensating joints to compensate for the errors in the position of apoint of interest may be limited by range of motion (ROM) limits of theone or more compensating joints. In some examples, when a ROM limit forone or more of the compensating joints is reached and/or is about to bereached, method 500 and/or process 570 may be stopped and an error maybe indicated to the operator using one or more visible and/or audibleerror cues. In some examples, rather than stopping operation of method500 and/or process 570, process 570 may operate in modified form topartially compensate for movements from a disturbance so as to minimizethe controllable error while providing feedback to the operator that notall of the movement caused by the disturbance is being compensated for.In some examples, the feedback may include one or more visible and/oraudio cues indicating that compensation is limited and/or theapplication of resistance on the one or more compensating joints. Insome examples, the resistance may include partially applying one or morebrakes associated with the one or more compensating joints and/orapplying a motion resistance voltage and/or signal in one or moreactuators associated with the one or more compensating joints.

As discussed above and further emphasized here, FIG. 5 is merely anexample which should not unduly limit the scope of the claims. One ofordinary skill in the art would recognize many variations, alternatives,and modifications. According to some embodiments, method 500 may beindependently applied for each of the instruments being manipulated bythe computer-assisted device. In some examples, the instruments mayinclude any of the instruments inserted through a body opening on thepatient. In some examples, the compensating joints may be located distalto an arm mounting platform, such as arm mounting platform 227, of thecomputer-assisted device so that compensation to maintain the placementof end effectors is applied separately for each of the end effectors.

According to some embodiments, the disturbed and compensating joints maynot include each of the joints in the articulated arm and/ormanipulator. In some examples, the compensating joints may include justthe roll, pitch, and yaw joints of the manipulator. In some examples,other joints in the articulated arm and/or the manipulator may be lockedto prevent their relative movement during method 500. In some examples,one or more non-actuated joints of the articulated arm and/or themanipulator may be unlocked and/or placed in a float state during method500 so that disturbances to the placement of the end effector may be atleast partially reduced by changes in the unlocked joints. In someexamples, the changes in the unlocked joints may reduce the amount thatthe compensating joints are to be driven. In some examples, the pose ofthe instrument may be at least partially maintained using resistancefrom the body wall at the insertion point of the instrument and/or by anoperator of the computer-assisted device.

According to some embodiments, one or more of the processes 530-570 maybe performed concurrently. According to some embodiments, additionalconditions may result in premature termination of method 500 such as byreturning control of the computer-assisted device to an operator and/orby suspension of operation of the computer-assisted device. In someexamples, the additional conditions may include inability to completethe compensated movement, manual intervention and/or override from anoperator using one or more controls on an operator workstation and/orthe articulated arms, detection of operator disengagement with theoperator workstation using one or more safety interlocks, positiontracking errors in the computer-assisted device, system faults, and/orthe like. In some examples, the desired movement may not be possible dueto the detection of imminent collisions among the links and/or joints ofthe computer-assisted device, range of motion limits in one or more ofthe joints of the computer-assisted device, inability to maintain thepose of the instrument due to motion of the patient, and/or the like. Insome examples, premature termination of method 500 may result in anerror notification being sent to the operator. In some examples, theerror notification may include any visual and/or audible indicator, suchas a text message, a blinking light, an audible tone, a spoken phrase,and/or the like.

During the disturbance of joints or compensation of disturbances, suchas the when implementing method 400 of FIG. 4 and/or 500 of FIG. 5, itmay be beneficial to still allow for teleoperated control of theinstruments by an operator. Teleoperated control allows the surgeon tomake small adjustments to counteract disturbances and/or in cases wheresome of the disturbances are not completely accounted for by thedisturbance compensation and/or when there is over compensation.Furthermore, a surgeon may continue with a procedure during adisturbance. To aid in the coordination of an operator controlling oneor more of the instruments of a system such as instrument 270 ofcomputer aided system 200 of FIG. 2, the system may set up the controlsystem to have an intuitive reference frame.

In some embodiments, an operator views the instruments of computer aidedsystem 200 of FIG. 2 through a display system such as display system 192of FIG. 1. The display system 192 may be a video stream from a camera,such as an endoscope, that is mounted as an instrument on an articulatedarm of computer aided system 200. The camera may display instrumentsfrom other articulated arms that may be controlled by controllers, suchas input controls 195 of FIG. 1. For the sake of intuitive controls andcommand of the instruments, the controllers may accept commands in thereference frame of the display, which may be the reference frame of theimaging device/video camera/endoscope.

In some embodiments, when driving joints for compensation or inaccordance with user control commands, the movements of the joints maybe bandwidth limited, velocity limited, bandwidth controlled, and/orvelocity controlled based on the configuration of the articulated armand the end effector in relation to the articulated arm. For example,with respect to FIG. 2, when end effector 276 is fully extended awayfrom instrument carriage 268, small motions and small velocity movementsof the arms from driving one or more joints will cause large movementsand faster movements at end effector 276. In contrast, when end effector276 is fully retracted, large motions and large velocity movements ofthe arms from driving one or more joints will translate to smallmovements and slower velocities at end effector 276. Similarly,depending upon how far forward the articulated arm is pitched forwardand/or back, yaw rotational movement and velocities will be magnifiedand/or scaled down.

In some embodiments, driving joints for compensation may be bandwidthlimited and/or velocity limited by breaking the compensation movementsinto several iterative parts. For example, 10 iterative parts over a 0.2second time span. In this manner, compensation joints may be preventedfrom conducting large movements in a very short period of time causingadditional disturbances to the disturbed joints. For example, when aninstrument is close to fully withdrawn, small compensating motions atthe end effector may require large motions at one or more of thecompensating joints. A fast response by one or more joints for a largemotion can jerk the disturbed joint causing an additional disturbanceand sometimes a feedback loop between disturbing the disturbed jointsduring compensation and then compensating for that disturbance, whichcauses another disturbance. Thus, depending on the orientation of one ormore joints and or the end effector, the joints may be velocity limited.In some embodiments, a hard velocity limit may be applied to the jointsin all configurations.

FIGS. 6A and 6B illustrates an exemplary camera view 600 from twodifferent perspectives. FIG. 6A is an overhead perspective, and FIG. 6Bis the perspective of a sensor of imaging device 610. Camera view 600from the perspective of FIG. 6B may be viewed from a display, such asdisplay system 192 of operator workstation 190 in FIG. 1, receivingstreaming image captures from imaging device 610. In some embodiments,imaging device 610 is an endoscope and controlled by an articulated arm,such as articulate arm 120 of FIG. 1 and/or the articulated arm of FIG.2. In FIG. 6A, camera view 600 is delineated by the dotted line whichmay represent an exemplary field of view and focus area for imagingdevice 610. In FIG. 6B, an exemplary camera view 600 is shown from theperspective of a user viewing a video stream from imaging device 610 ona display, such as display system 192 of operator workstation 190 ofFIG. 1. In some embodiments, the video stream provided by imaging device610 may be stereoscopic. Imaging device 610 may use one or more sensorsfor providing stereoscopic video streams. In this manner, the operatormay have a sense of depth perception when using a system such ascomputer aided system 100 of FIG. 1. Camera coordinate frame 611illustrates the coordinate frame of imaging device 610. In FIG. 6A,camera coordinate frame 611 shows the Z1 and Y1 axes of cameracoordinate frame 611 with the X1 axis (not shown) going in and out ofthe page. In FIG. 6B the X1 and Y1 axes of camera coordinate frame 611are shown with the Z1 axis (not shown) going in and out of the page. Insome embodiments, camera coordinate frame 611 may be camera coordinateframe 363 of FIG. 3.

FIGS. 6A and 6B also include instruments 620 and 630, which arecontrolled by one or more articulated arms, such as articulated arms 120of FIG. 1 and/or the articulated arm of FIG. 2. Instruments 620 and 630may be within camera view 600 and may be manipulated by one or moreusers or operators using controls, such as input controls 195 of FIG. 1,and viewing instruments 620 and 630 from the perspective of FIG. 6B.FIGS. 6A and 6B also illustrate coordinate frames 621 and 631 ofinstruments 620 and 630, respectively, from different perspectives. Insome examples coordinate frames 621 and 631 may be the same asinstrument coordinate frames 343 and 353 of FIG. 3.

Because a user teleoperating instruments 620 and 630 may be viewing theinstruments from the perspective of FIG. 6B of camera view 600, it maybe useful for user commands to be conducted in the camera referenceframe 611. Any commands provided in the camera coordinate frame 611 canbe translated to commands in the coordinate frames of the instruments(e.g. frames 621 and 631), by using a kinematic model, such as kinematicmodel 300 of FIG. 3. In this manner, up and down is in relation to thecamera view, which may be generally in line of the perspective of theuser. A user command to move instrument 620 or 630 up and down maytranslate to the instrument moving along the X1 axis of cameracoordinate frame 611. Similarly, user commands for other translationalmotions may follow the Y1 and Z1 axes of camera coordinate frame 611. Insome embodiments, commands for rotational motions, such as roll, pitch,and yaw, may also be translated from the camera coordinate frame 611 tothe coordinate reference frames of the instruments (e.g. frames 621 and631).

In some embodiments, camera coordinate frame 611 may be detached fromthe physical imaging device 610. This may be beneficial in someembodiments where the instrument motion is fixed with the cameracoordinate frame. For example, if the position of instruments 620 and630 were commanded in relation to the camera coordinate frame and thecamera coordinate frame was fixed to imaging device 610, undesirabledisturbances to imaging device 610 would translate into undesirabledisturbances to instruments 620 and 630. In some embodiments, a user mayhave the option to move and/or realign the camera coordinate frame withthe imaging device 610. In this manner when imaging device 610 straystoo far from camera coordinate frame 611, such as when instrumentmovements become less intuitive to the user, the user can reset and/orreposition the camera coordinate frame.

In some instances there may be disturbances to one or more joints ofmultiple arms affecting the instruments and/or imaging devices of eacharm. This may occur, for example, when brakes are released for severaljoints, such as the staggered brake release discussed in method 400and/or the brake release of method 500. Furthermore, during adisturbance, it may be desirable to allow a user to maintain intuitivecontrol and operation of one or more of the arms, instruments, and orimaging devices.

FIG. 7 illustrates an exemplary method 700 for maintaining intuitivecontrol of one or more instruments during a disturbance according tosome embodiments. In some examples, the disturbance may occur in one ormore joints of one or more articulated arms and camera of a computeraided system, such as computer aided system 100 of FIG. 1.

At a process 710, a camera coordinate frame is set at the position of animaging device. In some embodiments, this is sometimes referred to as“latching” or being “latched.” The imaging device may be controlledand/or held by an articulated arm, such as articulated arm 120 of FIG. 1and/or the articulated arm of FIG. 2. In some embodiments, a cameracoordinate frame may be set/latched by recording a transform between acoordinate reference frame to the camera coordinate reference frame at aparticular instant of time, such as before a brake release and/or theintroduction of a disturbance. In some embodiments, the transform may bedetermined by using a kinematic model, such as model 300 of FIG. 3. Inaccordance with FIG. 3, the camera coordinate frame may be cameracoordinate frame 363 and the reference frame may be arm mountingplatform coordinate frame 330. The recorded transform may be a transformfrom the reference frame to the camera coordinate frame at a particularpoint in time with a particular configuration of the articulated armcontrolling the imaging device. In some embodiments, recording thetransform may include storing the transformation and/or kinematicrelationships between the camera coordinate frame and referencecoordinate frame on a computer readable medium, such as memory 150 ofFIG. 1.

At a process 720, disturbances may be allowed and/or introduced into thesystem. For example one or more brakes for one or more joints to one ormore articulated arms may be released. This may include the release ofbrakes on one or more joints of the articulated arm controlling theimaging device and/or instruments. In some embodiments, the disturbancesmay be caused by the staggered brake release of method 400 and/or thebrake release of method 500.

At a process 730, disturbances to the instruments and/or imaging deviceare compensated for such that the movement of the end effectors andimaging device caused by the disturbance is reduced and/or eliminated.In some embodiments, the compensation conducted at process 730 for eachinstrument and imaging device may be conducted using one or more of theprocesses in method 500 of FIG. 5. In some embodiments, the imagingdevice may be let left alone and allowed to be disturbed withoutcompensation.

At a process 740, the computer aided system may receive instrumentmotion commands. The instrument motion commands may come from a usermanipulating controls, such as the input controls 195 of FIG. 1. Theinstrument motion commands may be concurrently with a disturbance.

At a process 750, the commands received at process 740 are transformedfrom the camera coordinate frame recorded during process 710 to thecoordinate frame of the respective instruments using the transformationrecorded/stored during process 710. In some embodiments the physicalimaging device represented by the camera coordinate frame may have beendisturbed and moved away, and therefore, no longer in the same positionas the camera coordinate frame recorded during 710. In some examples,this difference may result in a reduced level of intuition in control ofthe instruments, which may be corrected at any time by having theoperator reposition the camera using the articulated arm to which it isattached and/or resetting the camera coordinate frame.

At a process 760, the computer aided system drives the joints to movethe instruments in accordance with the commands transformed from thecamera coordinate frame into the instrument reference frame at process750. According to some embodiments, one or more of the processes 710-760may be performed concurrently.

In some embodiments, the process of 760 can occur concurrently with thecomputer aided system driving the joints to compensate for disturbances.For example, user commands driving the joints may be superimposed ontothe commands to drive the joint based on disturbance compensations. Themovement of the joints, as discussed above, may be bandwidth limited,velocity limited, bandwidth controlled, and/or velocity controlled.Driving the joints based on user commands super imposed on top ofcompensating commands may be controlled and/or limited in a similarmanner as in the examples discussed above in relation to FIGS. 4 and 5.

Although illustrative embodiments have been shown and described, a widerange of modification, change and substitution is contemplated in theforegoing disclosure and in some instances, some features of theembodiments may be employed without a corresponding use of otherfeatures. One of ordinary skill in the art would recognize manyvariations, alternatives, and modifications. Thus, the scope of theinvention should be limited only by the following claims, and it isappropriate that the claims be construed broadly and in a mannerconsistent with the scope of the embodiments disclosed herein.

What is claimed is:
 1. A computer-assisted medical device comprising: afirst articulated arm with a first plurality of joints; and a controlunit coupled to the first articulated arm, wherein the control unit isconfigured to: send a first command to a first plurality of brakes inthe first articulated arm to begin a release of the first plurality ofbrakes in a first predetermined staggered manner at calculated intervalsthat prevent simultaneous release of the first plurality of brakes. 2.The device of claim 1, wherein the first predetermined staggered mannercauses brakes in the first plurality of brakes to release within apredetermined time of each other.
 3. The device of claim 1, furthercomprising: a second articulated arm; and wherein the control unit isfurther configured to: send a second command to a second plurality ofbrakes in the second articulated arm to begin the release of the secondplurality of brakes in a second predetermined staggered manner, whereinthe first and second commands assign first and second timing windows forbeginning the release of the first and second plurality of brakes. 4.The device of claim 3, wherein: the first and second timing windows arebased on a number of clock cycles within a time period; and the firstand second articulated arms each determine a current time window bytaking a modulus of a global clock with the number of clock cycleswithin the time period.
 5. The device of claim 3, wherein the first andsecond commands assign first and second timing windows for beginning therelease of the first and second plurality of brakes in a manner suchthat no brake of the first and second plurality of brakes is released ata same time as another brake of the first and second plurality ofbrakes.
 6. The device of claim 1, wherein the first predeterminedstaggered manner causes each brake in the first plurality of brakes tobegin a gradual release within a predetermined time of other brakes ofthe first plurality of brakes.
 7. The device of claim 1, wherein thefirst predetermined staggered manner releases the brakes in the firstplurality of brakes in an order such that: a vertical joint is releasedlast; or a greatest disturbance is caused by a last brake release. 8.The device of claim 1 wherein the control unit is further configured tomove the first articulated arm during the release of the first pluralityof brakes.
 9. A method of controlling motion in a medical device, themethod comprising: sending, by a control unit of the medical device, afirst command to a first plurality of brakes in a first articulated armto begin a release of the first plurality of brakes in a firstpredetermined staggered manner at calculated intervals that preventsimultaneous release of the first plurality of brakes.
 10. The method ofclaim 9, wherein the first predetermined staggered manner causes brakesin the first plurality of brakes to release within a predetermined timeof each other.
 11. The method of claim 9, further comprising sending asecond command to a second plurality of brakes in a second articulatedarm to begin a release of the second plurality of brakes in a secondpredetermined staggered manner, wherein the first and second commandsassign a first and second timing window for beginning the release of thefirst and second plurality of brakes.
 12. The method of claim 11, themethod further comprising sending a global clock to the first and secondarticulated arms, wherein the first and second timing windows are basedon a number of clock cycles within a time period, and wherein a currenttiming window can be determined by taking a modulus of the global clockwith the number of clock cycles within the time period.
 13. The methodof claim 11, wherein the first and second commands assign first andsecond timing windows for beginning the release of the first and secondplurality of brakes in a manner such that no brake of the first andsecond plurality of brakes will release at a same time as another brakeof the first and second plurality of brakes.
 14. The method of claim 9,wherein the first predetermined staggered manner causes each brake inthe first plurality of brakes to begin a gradual release within apredetermined time of other brakes of the first plurality of brakes. 15.The method of claim 14, wherein the first command comprises instructionsfor the gradual release within a predetermined time using a rampedchange in voltage or current.
 16. The method of claim 9, wherein thefirst predetermined staggered manner releases the brakes in the firstplurality of brakes in an order such that a greatest disturbance iscaused by a last brake release or a vertical joint is released last. 17.The method of claim 9, further comprising: receiving a user command tomove the first articulated arm; and moving the first articulated armbased on a user command during the release of the first plurality ofbrakes.
 18. A non-transitory machine-readable medium comprising aplurality of machine-readable instructions which when executed by one ormore processors associated with a medical device are adapted to causethe one or more processors to perform a method comprising: sending afirst command to a first plurality of brakes in a first articulated armto begin a release of the first plurality of brakes in a firstpredetermined staggered manner at calculated intervals that preventsimultaneous release of the first plurality of brakes.
 19. Thenon-transitory machine-readable medium of claim 18, wherein the firstpredetermined staggered manner causes brakes in the first plurality ofbrakes to release within a predetermined time of each other.
 20. Thenon-transitory machine-readable medium of claim 18, wherein the methodfurther comprises sending a second command to a second plurality ofbrakes in a second articulated arm to begin the release of the secondplurality of brakes in a second predetermined staggered manner, andwherein the first and second commands assign a first and second timingwindow for beginning the release of the first and second plurality ofbrakes.
 21. The non-transitory machine-readable medium of claim 20,wherein the first and second commands assign first and second timingwindows for beginning the release of the first and second plurality ofbrakes in a manner such that no brake of the first and second pluralityof brakes will release at a same time as another brake of the first andsecond plurality of brakes.
 22. The non-transitory machine-readablemedium of claim 18, wherein the first predetermined staggered mannerreleases the brakes in the first plurality of brakes in an order suchthat a greatest disturbance is caused by a last brake release or avertical joint is released last.